PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them

Abstract Open-domain Question Answering models that directly leverage question-answer (QA) pairs, such as closed-book QA (CBQA) and QA-pair retrievers, show promise in terms of speed memory compared with conventional which retrieve read from text corpora. retrievers also offer interpretable answers, a high degree control, are trivial to update at test time new knowledge. However, these fall short the accuracy retrieve-and-read systems, substantially less knowledge is covered by available QA-pairs relative corpora like Wikipedia. To facilitate improved models, we introduce Probably Asked Questions (PAQ), very large resource 65M automatically generated QA-pairs. We retriever, RePAQ, complement PAQ. find PAQ preempts caches questions, enabling RePAQ match recent whilst being significantly faster. Using PAQ, train CBQA outperform comparable baselines 5%, but trail over 15%, indicating effectiveness explicit retrieval. can be configured for size (under 500MB) or (over 1K questions per second) while retaining accuracy. Lastly, demonstrate RePAQ’s strength selective QA, abstaining answering when it likely incorrect. This enables “back-off” more expensive state-of-the-art model, leading combined system both accurate 2x faster than model alone.